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IDENTIFYING  FUTURE  TRAINING  TECHNOLOGY  OPPORTUNITIES 


USING  CAREER  FIELD  MODELS  AND  SIMULATIONS 

INTRODUCTION 

In  response  to  the  recent  U.S.  Quadrennial  Defense  Review  (QDR),  the  U.S.  Military 
Services  are  taking  an  unprecedented  top-down  look  at  their  entire  job  and  training  infrastructure. 
The  Services  are  attempting  to  identify  ways  to  significantly  reduce  the  resources  required  for 
providing  education  and  training  development  and  delivery.  The  options  being  considered  include 
privatizing  education  and  training  using  the  civilian  training  establishment,  eliminating  courses 
altogether,  and  applying  advanced  training  technologies  to  in-residence  and  field  training  to  reduce 
the  time  required  for  training  and  to  increase  the  efficiency  and  effectiveness  of  the  process.  At 
the  same  time,  the  Military  has  been  challenged  by  the  QDR  to  continue  to  provide  career  learning 
opportunities  for  personnel.  One  of  the  most  pressing  issues  is  related  to  identifying  the  most  cost- 
effective  places  within  the  education  and  training  system  for  applying  one  or  more  of  the  options. 
Further,  decision  makers  must  evaluate  these  options  by  examining  the  life-cycle  costs  associated 
with  each  option  within  and  across  military  career  fields. 

Given  recent  reductions  in  manpower,  personnel,  and  training  (MPT)  and  the  continued 
pressure  to  "rightsize"  the  Air  Force  to  meet  current  defense  demands,  finding  effective  ways  to 
assess  the  impact  of  education  and  training  options  becomes  especially  critical.  Conventional 
approaches  to  assessing  the  value-added  or  benefit  of  human  resources  interventions  and 
organizational  change  have  attempted  to  relate  the  intervention  directly  to  measures  of 
organizational  productivity  in  a  traditional  utility  analysis  approach  (e.g.,  Cascio,  1989).  In 
general,  utility  analysis  uses  an  estimate  of  the  validity  of  a  personnel  intervention,  such  as  a 
personnel  selection  method  or  a  training  program,  and  translates  this  parameter  into  organizational 
productivity  in  terms  of  dollars.  However,  certain  aspects  of  implementing  utility  approaches  have 
presented  problems  (e.g.,  Greer  &  Cascio,  1987).  In  addition,  utility  analysis  may  be  quite  limited 
as  a  method  of  determining  the  benefit  one  might  actually  expect  from  a  training  program  if  the 
effects  of  suspected  moderators  cannot  be  conceptualized  easily  with  respect  to  the  training 
program  (Cascio,  1989). 
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For  example,  the  Air  Force  training  community  may  be  interested  in  identifying  the 
significant  cost  drivers  associated  with  implementing  and  maintaining  advanced  technology 
training  programs.  In  addition,  they  might  be  interested  in  the  extent  to  which  course  could  be 
refined,  shortened,  or  exported  to  the  field  using  advanced  training  technologies.  In  many 
instances,  implementing  some  type  f  training  technology  may  not  be  immediately  or  near-term  cost 
effective.  By  implementing  changes  such  as  these,  we  have  impacted  our  previous  estimates  of  the 
value-added  or  utility  of  the  previous  course  or  training  pipeline.  This  is  due  to  the  fact  that  there 
may  be  significant  up-front  costs  associated  with  procuring  equipment  to  support  the  new 
technology.  In  addition,  there  are  inevitable,  periodic,  updates  to  the  courseware  and  to  the 
support  infrastructure.  There  are  also  somewhat  less  tangible  costs  associated  with  potential 
changes  in  the  quality  of  graduates  once  the  technology-based  course  has  been  implemented. 
Therefore,  new  validation  studies  and  examinations  must  be  undertaken  when  changes  such  as 
these  are  made. 

This  report  argues  that  the  use  of  probability-based  organizational  simulation  provides  a  more 
flexible  method  for  assessing  intervention  utility  under  a  variety  of  types  of  organization 
reengineering.  The  ability  of  training  utility  analysis  to  reflect  organizational  impact  will  be 
further  eroded  if  Cascio’s  (1995)  predictions  about  the  fluidity  of  work  and  the  continued 
integration  of  technology  in  the  next  generation  of  organizations  are  realized. 

A  promising  and  recently  proven  alternative  approach  to  the  assessment  of  the  impact  of 
change  involves  the  use  of  a  probability-based,  organizational  simulation  technology  such  as  the 
Training  Impacts  Decision  System,  or  TIDES;  (see  Mitchell,  Yadrick,  &  Bennett;  1993;  Vaughan 
&  Yadrick,  1992).  TIDES  was  developed  by  the  U.S.  Air  Force  for  assessing  the  effects  of  these 
activities.  The  simulation  has  the  analytic  capability  to  (a)  provide  a  measure  of  training  utility, 
expressed  as  changes  in  overall  training  costs  and  changes  in  the  requirements  for,  and  availability 
of,  qualified  personnel;  (b)  quantify  the  reduction  in  requirements  such  as  travel-to-school  for 
additional  training  in  some  areas;  and  (c)  estimate  the  “life-cycle”  costs  and  benefits  of  change  in  a 
variety  of  tenns  of  relevance  to  senior  managers. 
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The  TIDES  model  relates  micro-level  personnel  and  training  events  to  macro-level  outcome 
variables.  Data  for  TIDES  comes  from  a  variety  of  sources,  including  job  and  training  analysis, 
existing  MPT  data  bases,  and  subject-matter  expert  judgments.  A  computer  simulation  provides 
information  on  the  flow  of  individuals  through  jobs,  task  performance  requirements,  and  various 
formal  and  infonnal  (e.g.,  on-the-job)  training  requirements.  From  these  individual  events,  the 
system  estimates  task-level,  on-the-job  training  events.  Finally,  the  system  estimates  overall 
training  resource  requirements,  costs,  and  capacities  from  the  task-level  events.  Once  the 
computer  simulation  of  the  current  flows  (the  baseline)  has  been  developed,  plausible  alternative 
flows  and  job  and  training  structuring  can  be  developed  and  new  simulations  can  be  conducted. 
Results  from  the  alternative  simulation  outcomes  can  be  compared  to  the  baseline  to  examine 
impacts  associated  with  the  alternatives. 

METHOD 

The  Air  Force  Office  of  the  Surgeon  General  requested  analyses  to  determine  if  developing 
exportable  training  courses,  based  in  the  in-residence  course  currently  being  provided,  could 
reduce  the  travel-to-school  costs  and  requirements  for  Reserve  medical  personnel,  while 
maintaining  their  task-level  proficiency.  The  goal  of  the  request  was  to  quantify  the  impact  and  to 
determine  if  sufficient  resources  would  be  saved  to  “pay  for”  the  development  and  delivery  of 
training  in  an  exportable  medium.  This  study  conducted  a  series  of  comparative  analyses  across  a 
range  of  career  fields  and  in-residence  courses  to  examine  the  costs  and  benefits  of  technology 
insertion  for  reducing  travel-to-school  costs  while  maintaining  current  proficiency  levels.  The 
chosen  career  fields  were  Aerospace  Physiology;  Electronics,  Computer,  &  Switching  Systems; 
and  Aerospace  Propulsion.  Table  1  presents  information  on  the  student  flow  and  size  of  each 
career  field’s  training  courses. 
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TABLE  1 .  Summary  Information  for  Each  Career  Field  and  Courses 


Key  Information 

Aerospace  Physiology 

Electronics,  Computers  & 
Switching  Systems 

Aerospace  Propulsion 

Average  Force  Level 

518 

2,774 

10,412 

Average  Trainees  Per  Year 

154 

1,266 

2,141 

Accessions  Per  Month 

4 

34 

60 

Number  of  Initial  Skills  Courses 

1 

10 

4 

Flours  of  Initial  Skills  CBT 

122 

1,588 

202 

Average  Number  of  Trainees  in 
Upgrade  Training  Per  Year 

20 

26 

107 

Number  of  Upgrade  Training 
Courses 

3 

4 

2 

Flours  of  Exportable  Upgrade 
CBT 

338 

530 

49 

PROCEDURE 

Although  space  limitations  preclude  a  detailed  discussion  of  the  mechanics  of  the  process 
here,  a  full  description  of  this  simulation,  including  the  essential  assumptions,  is  available  from  the 
first  author.  Several  assumptions  about  the  nature  of  technology  insertion  were  made  so  that 
changes  to  the  underlying  models  could  be  developed  and  implemented.  For  each  of  the  career 
field  chosen  for  comparative  study,  it  was  assumed  that  roughly  70%  of  the  current  stand-up 
lecture-based  in-residence  and  upgrade  training  courses,  could  be  delivered  in  a  computer-based 
mode  in  30%  less  time.  For  modeling  purposes  then,  70%  of  the  in-residence  and  the  upgrade 
training  course  time  was  reduced  an  average  of  30%,  i.e.,  70%  of  the  course  was  taught  in  30% 
less  time,  reducing  course  length  by  21%  on  average.  The  reduction  in  course  length  results  in 
training  dollar  savings  since  airmen  are  in  residence  for  the  course  for  a  shorter  duration  of  time, 
or  not  on  duty  during  training  for  upgrade  courses.  In  addition,  it  was  assumed  that  the  existing 
proficiency  levels  produced  by  the  changed  content  areas  would  be  constant.  In  other  words,  the 
level  of  proficiency  that  was  being  produced  by  the  courses  would  be  the  same  after  the  change,  as 
they  were  before  the  change. 


An  estimate  of  the  costs  of  purchasing  and  implementing  a  computer-based  training 
classroom  to  support  the  new  course  was  added  as  an  additional  training  cost  for  the  initial  year  of 
the  new  course.  These  costs  were  assumed  to  require  the  purchase  of  computer  hardware  systems 
for  the  in-residence  initial  skills  courses.  It  was  assumed  that  the  number  of  systems  purchased 

would  be  based  upon  the  number  of  new  airmen  entering  initial  skills  courses  for  the  AFS  being 
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considered.  For  upgrade  training,  it  was  assumed  that  no  computer  hardware  would  be  purchased. 
It  was  assumed  that  all  upgrade  training  would  occur  at  the  base  to  which  the  airman  was  assigned 
and  on  existing  computer  hardware.  Further,  in  subsequent  years,  costs  for  maintenance  and 
upgrade  to  course  content  and  hardware  maintenance  were  included.  No  hardware  maintenance 
costs  were  included  for  upgrade  training  since  training  would  be  performed  on  pre-existing 
hardware  systems.  Assumptions  were  also  made  concerning  travel-to-school  requirements.  Initial 
skills  courses  would  remain  in  residence  so  no  change  would  be  made  in  existing  travel-to-school 
requirements.  Upgrade  training  would  now  be  perfonned  at  the  airman’s  current  base  assignment, 
therefore  eliminating  all  travel-to-school  requirements  for  upgrade  training.  Table  2  provides  a 
summary  of  these  additional  assumptions  and  cost  figures. 


TABLE  2.  Additional  Assumptions 


Cost  Category 

Implementation  of  Initial  Skills  Cours 
CBT 

Implementation  of  Exportable  Upgrad 
CBT 

Acquisition  Costs 

■  Hardware 

$2,000  per  Monthly  Accession 

$0 

■  Course  Development 

$50  per  Development  Hour. 

300  Development  Hours  per  Hour  of 
CBT 

$50  per  Development  Hour. 

300  Development  Hours  per  Hour  of 
CBT 

■  Maintenance 

$50  per  3  Hours  of  Delivered  CBT 

$0 

Travel-to-School  Requirements 

■  Initial  Skills  Training 

No  Change 

■  Upgrade  Training 

Eliminated 

RESULTS 

Figure  1  shows  the  results  of  our  simulation  and  the  impact  of  developing  an  exportable 
upgrade  course  using  CBT  for  each  career  field.  As  the  figures  for  each  career  field  demonstrate, 
there  are  savings  in  travel-to-school  costs  associated  with  this  specific  change,  but  only  for  the 
career  fields  with  the  lager  student  flows  and  shortest  upgrade  training  course  (e.g.,  Aerospace 
Propulsion  and  Electronics,  Computers,  &  Switching,  respectively).  Also  note  that  the  simulation 
results  permit  assessments  of  the  time  required,  in  years,  before  there  is  a  return  on  the  investment 
in  this  type  of  training.  In  the  present  analysis,  there  is  “break  even”  point  at  about  three  years  and 
savings  in  subsequent  years,  especially  in  Aerospace  Propulsion. 
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Figure  1.  Career  Field  Summary  Analyses  of  Total  Training  Costs  for  Exportable  Upgrade  Training 
Courses 
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The  next  step  in  the  analysis  was  to  assess  the  combined  effect  of  using  CBT  in  the  initial 
skills  and  in  the  upgrade  training  for  each  of  the  career  fields.  Figure  2  presents  the  results  from 
these  combined  analyses.  Again,  these  analyses  demonstrate  that  there  are  savings  to  be 
realized  from  the  conversion  of  both  the  initial  skills  and  upgrade  training  course  for  the  two 
larger  career  fields  -  Aerospace  Propulsion  and  Electronics,  Computers  and  Switching  -  but  not 
for  Aerospace  Physiology. 

Figure  2.  Career  Field  Summary  Analyses  of  Total  Training  Costs  for  Initial  Skills  and 
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The  final  analyses  were  conducted  to  determine  the  savings,  if  any,  on  travel-to-school 


costs.  As  mentioned  earlier,  these  costs  were  of  special  interest  to  the  Air  Force  Surgeon 
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General  as  a  means  of  assessing  a  potential  benefit  of  CBT  for  exportable  training  in  the  future. 
Figure  3  provides  the  results  of  these  analyses. 


Figure  3.  Career  Field  Summary  Analyses  of  Travel-to-School  Costs 
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DISCUSSION 

The  results  indicate  that  there  are  potential  savings  to  be  realized  from  the  introduction  of 
advanced  training  technology  into  these  courses.  However,  these  analyses  demonstrate  that 
there  are  some  key  characteristics  of  the  underlying  courses  and  the  size  of  the  career  field  that 
dictate  the  extent  of  any  savings  of  benefit  impact  from  change.  For  example,  there  is  clear 
utility  from  the  application  of  CBT  for  Aerospace  Propulsion.  This  career  field  is  also  one  of 
the  largest  in  terms  of  student  flow,  and  of  moderate  size  with  respect  to  the  number  of  hours  to 
be  converted.  In  comparison,  Electronic,  Computers,  and  Switching  has  a  number  of  courses  to 
be  converted,  the  number  of  hours  associated  with  them  is  quite  large,  and  although  the  student 
flow  is  fairly  high,  the  combination  of  these  factors  limits  the  short-tenn  savings  to  be  realized 
from  CBT.  In  contrast,  Aerospace  Physiology  is  a  relatively  small  career  field  with  low  student 
flow.  Although  it  has  a  small  number  of  course  hours  to  be  converted  and  although  it  has  few 
courses  overall,  there  are  very  little  if  any  savings  to  be  gained  by  applying  CBT  to  this  career 
field. 

It  is  also  important  to  note  that  one  of  the  major  points  of  this  study  was  to  demonstrate  the 
potential  use  of  simulations  for  identifying  cost-effective  opportunities  for  technology  insertion. 
As  the  results  indicate,  organizational  simulations  offer  considerable  insight  and  data  for 
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decision  making  in  support  of  training  restructuring  and  change.  In  addition,  the  simulations 
can  be  developed  and  executed  before  budgetary  and  related  resources  decisions  are  made.  This 
can  ensure  a  more  efficient  and  effective  use  of  the  Military’s  shrinking  resources  for  training. 

In  summary,  the  analyses  presented  here  were  provided  to  the  Air  Force  Office  of  the 
Surgeon  General.  After  careful  review  of  these  results,  it  was  determined  that  CBT  could  be 
implemented,  in  a  systematic  way,  to  realize  travel-to-school  savings  without  sacrificing  the 
proficiency  of  the  graduates  from  the  training.  Additional  analyses  are  being  conducted  with 
the  U.S.  Air  Force  Education  and  Training  Command  to  identify  areas  where  maintenance 
training  costs  can  be  substantially  reduced  using  advanced  training  technologies. 
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